Exponential asymptotic optimality of Whittle index policy

We evaluate the performance of Whittle index policy for restless Markovian bandit. It is shown in Weber and Weiss (J Appl Probab 27(3):637–648, 1990) that if bandit indexable associated deterministic system has a global attractor fixed point, then asymptotically optimal regime where arm population grows proportionally with number activation arms. In this paper, we show that, under same conditions, convergence rate exponential population, unless point singular (to be defined later), which almost never happens practice. Our result holds continuous-time model (1990) discrete-time all bandits make synchronous transitions. proof based on nature equation governing stochastic system: it piecewise affine continuous dynamical inside simplex empirical measure Using simulations numerical solvers, also investigate cases, as well how level singularity influences (exponential) rate. illustrate our theorem fading channel model.